Papers with ensemble learning
Bag of Tricks for In-Distribution Calibration of Pretrained Transformers (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models (PLMs) often predict over-confidently. |
| Approach: | They propose to use ensemble learning and data augmentation to improve confidence calibration for PLMs by combining calibration techniques with a trade-off between accuracy and classification. |
| Outcome: | The proposed calibration method improves classification accuracy and confidence in pre-trained language models by combining several calibration techniques. |
Resource of Wikipedias in 31 Languages Categorized into Fine-Grained Named Entities (2022.coling-1)
Copied to clipboard
| Challenge: | a resource of Wikipedias in 31 languages is categorized into Extended Named Entity (ENE) ENE version 8 has 219 fine-grained NE categories. |
| Approach: | They describe a resource of Wikipedias in 31 languages categorized into Extended Named Entity (ENE) they first categorized 920 K Japanese Wikipedia pages using machine learning, then shared a task of Wikipedia categorization into 30 languages . |
| Outcome: | The proposed system is based on a dataset of Japanese Wikipedia pages . the dataset shows the best performance among the 30 languages . |
LLM-Forest: Ensemble Learning of LLMs with Graph-Augmented Prompts for Data Imputation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for missing data imputation are lacking in a finetuning-free process and mitigating biases and uncertainty in LLM outputs. |
| Approach: | They propose a framework for imputation of large language models with a forest of few-shot learning LLM "trees" they use bipartite information graphs to identify relevant neighboring entries with feature and value granularity. |
| Outcome: | The proposed framework is based on a concept of bipartite information graphs to identify high-quality relevant neighboring entries with both feature and value granularity. |
Ensembles of Hybrid and End-to-End Speech Recognition. (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to combine hybrid and end-to-end ASRs with confidence measures are limited and neither can achieve optimal performance. |
| Approach: | They propose to combine the hybrid Kaldi-based Automatic Speech Recognition system with the end-to-end wav2vec 2.0 XLS-R ASR using confidence measures. |
| Outcome: | The proposed method reduces the word error rate by 14% on the primary test set and 20% on other noisy and imbalanced data. |
One-Teacher and Multiple-Student Knowledge Distillation on Sentiment Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing knowledge distillation models require large computing resources and long inference time to perform. |
| Approach: | They propose a one-teacher and multiple-student knowledge distillation approach to distill a deep pre-trained teacher model into multiple shallow student models with ensemble learning. |
| Outcome: | The proposed method achieves better results with fewer parameters and extremely high speedup ratios on three sentiment classification tasks. |